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IN THE CLAIMS : 

Please cancel Claims 7 and 8 without prejudice or disclaimer of the subject 
matter recited therein. 

Please amend Claims 3-6 and add new Claim 9 as follows. 

1 . (Original) An image processing method of presenting a virtual object 
superimposed on a physical space to an observer, characterized by comprising: 

a holding step of holding, in a memory, information of a state of each 
virtual object included in a virtual space; 

an association step of associating at least one virtual object included in 
the virtual space with at least one position/orientation sensor which is held in a hand of the 
observer and manipulated; 

a layout step of laying out, in the virtual space, the virtual object 
associated with the position/orientation sensor in the association step in accordance with a 
position and orientation of the position/orientation sensor itself, which are determined on the 
basis of a measurement result by the position/orientation sensor; 

a presentation step of presenting the state of each virtual object 
included in the virtual space to the observer on the basis of the information held in the holding 
step; 

an input step of inputting an operation to change a desired one of the 
states presented in the presentation step; and 
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an updating step of updating the information in accordance with the 
operation input in the input step. 

2. (Original) The method according to claim 1, characterized by further 
comprising a registration step of registering, in the information, the position and orientation of 
the position/orientation sensor itself, which are determined on the basis of the measurement 
result by the position/orientation sensor as a position and orientation of the virtual object 
associated with the position/orientation sensor in the association step, 

wherein in the layout step, the virtual object is laid out in the virtual 
space by using the position and orientation of the virtual object registered in the information in 
the registration step. 

3. (Currently Amended) The method according to claim 1 oH£, 
charact e rized in that wherein the information includes a name, blinking state, selected/unselected 
state, and section display state of the virtual object. 

4. (Currently Amended) An image processing method,, including the steps 

of: 

acquiring position and orientation information of an observer, 
acquiring an image of a physical space; 
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generating an image of a virtual space in accordance with the position 
and orientation information of the observer-artd; 

compositing the image of the physical space and the image of the 
virtual space and displaying the composited image on a head mounted display worn by a user;; 
characterized by comp r ising : 

generating an operation panel image and compositing the operation 
panel image with the image of the physical space and the image of the virtual space; 

acquiring position information of an operation unit operated by the 

observer; and 

updating the operation panel image in accordance with a positional 
relationship between the operation panel image and the operation unit, 

wherein in the operation panel image, a part selected by the operation 

unit is enlarged. 

5. (Currently Amended) The method according to claim 4, cha r act e rized 

in that wherein 

the image of the virtual space is generated on the basis of a 3D CAD 

data of a virtual object, 

the operation panel contains an assembly tree based on the 3D CAD 

data, and 
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the enlarged part includes a component name contained in the assembly 

tree. 

6. (Currently Amended) An image processing apparatus for presenting a 
virtual object superimposed ort in a physical space to an observer, characterized by comprising: 

a holding unit adapted to hold information of a state of each virtual 
object included in a virtual space; 

an association unit adapted to associate at least one virtual object 
included in the virtual space with at least one position/orientation sensor manipulated by the 
observer; 

a layout unit adapted to lay out, in the virtual space, the virtual object 
associated with the position/orientation sensor by said association unit in accordance with a 
position and orientation of the position/orientation sensor itself, which are determined on the 
basis of a measurement result by the position/orientation sensor; 

a presentation unit adapted to present the state of each virtual object 
included in the virtual space to the observer on the basis of the information held by said holding 
unit; 

an input unit adapted to input an operation to change a desired one of 
the states presented by said presentation unit; and 

an updating unit adapted to update the information in accordance with 
the operation input by said input unit. 
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Claims 7 and 8. (Cancelled). 

9. (New) A computer-readable medium encoded with a computer 
program for causing a computer to execute an image processing method of presenting a virtual 
object superimposed on a physical space to an observer, comprising: 

a holding step of holding, in a memory, information of a state of each 
virtual object included in a virtual space; 

an association step of associating at least one virtual object included in 
the virtual space with at least one position/orientation sensor which is held in a hand of the 
observer and manipulated; 

a layout step of laying out, in the virtual space, the virtual object 
associated with the position/orientation sensor in the association step in accordance with a 
position and orientation of the position/orientation sensor itself, which are determined on the 
basis of a measurement result by the position/orientation sensor; 

a presentation step of presenting the state of each virtual object 
included in the virtual space to the observer on the basis of the information held in the holding 
step; 

an input step of inputting an operation to change a desired one of the 
states presented in the presentation step; and 

an updating step of updating the information in accordance with the 
operation input in the input step. 



